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Abstract — A multiplicative Gaussian wire-tap channel inspired 
by compressed sensing is studied. Lower and upper bounds on 
the secrecy capacity are derived, and shown to be relatively tight 
in the large system limit for a large class of compressed sensing 
matrices. Surprisingly, it is shown that the secrecy capacity of 
this channel is nearly equal to the capacity without any secrecy 
constraint provided that the channel of the eavesdropper is 
strictly worse than the channel of the intended receiver. In other 
words, the eavesdropper can see almost everything and yet learn 
almost nothing. This behavior, which contrasts sharply with that 
of many commonly studied wiretap channels, is made possible 
by the fact that a small number of linear projections can make 
a crucial difference in the ability to estimate sparse vectors. 

I. Introduction 

Following Shannon's theory in 1949 of information- 
theoretic secrecy flj, Wyner introduced the wiretap channel 
in 1975 J2) . In the wiretap setting, a sender Alice wishes to 
communicate a message to a receiver Bob over a main channel 
but her transmissions are intercepted by an eavesdropper 
Eve through a secondary wiretap channel. The present paper 
analyzes a multiplicative Gaussian wiretap channel inspired 
by compressed sensing. The input to the channel is a p-length 
binary vector. The channel output is a linear transform of 
the input after it has first been corrupted by multiplicative 
white Gaussian noise. We analyze the setting where Bob and 
Eve observe different linear transforms characterized by two 
different channel matrices. 

Secrecy via compressed sensing schemes has received little 
attention from an information-theoretic viewpoint. In prior 
work, authors consider using a sensing matrix as a key (un- 
known to the eavesdropper) for both encryption and compres- 
sion [3]. Privacy via compressed sensing and linear program- 
ming decoding was explored in |4). By contrast, this paper 
assumes that the sensing matrices are known (non-secret); 
as a special case, Eve's sensing matrix might correspond to 
a subset of the rows of Bob's channel matrix. Our analysis 
shows that certain channel matrices, inspired by compressed 
sensing, allow for secrecy rates that are nearly equal to the 
main channel capacity even if Eve's capacity is large. 

A. Channel Model 

Outlined in Fig. Q] the multiplicative Gaussian wiretap 
channel with binary vector input is characterized by 

Y = A b WX, (1) 
Z = A e WX, (2) 

tAl so with the School of Computer and Communication Sciences, EPFL, 
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Fig. I. (Multiplicative Gaussian Wiretap Channel) For each block length n, 
Alice transmits a sequence of n binary valued support vectors X £ {0, 1} P 
over a main channel characterized by a matrix transform so that Bob receives 
Y = AijW^i.. The eavesdropper Eve receives Z = A £ W^- 



where X 6 {0, 1} P is the transmitted signal, and Y 6 K mfc , 
Z £ M" 1 " are the received real-valued signals at the legitimate 
user and eavesdropper, respectively. A related channel model 
in (5 1 also involves a wire-tap setting with binary input and 
real-valued output. The dimensions of the channel satisfy < 
m e < nib < p/2. The linear mixing parameters, matrices Af, G 
jn>m(,x P an( j ^ g R m e x P ? are fixed and known to all parties. 
The randomness of the channel is derived from W G M pxp , 
a diagonal matrix whose values are i.i.d. Gaussian random 
variables with mean zero and variance one. The channel is 
assumed to be memoryless between channel uses. 

B. Secrecy Capacity 

Alice selects a message S n S [1 : 2 n P R ], where R represents 
a normalized rate, and wishes to communicate reliably with 
Bob while keeping the message secret from Eve. A (2 npR ,n) 
secrecy code for the multiplicative wiretap channel consists of 
the following: (1) A message set [1 : 2 npR ]; (2) A randomized 
encoder that generates a codeword X n (S n ), S n G [1 : 2 npR ], 
according to Px. n \s„> 0) A decoder that assigns a message 
S„(Y") to each received sequence Y" G y n . The message 
S n is a random variable with entropy satisfying 

H{S; 



lim 
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A secrecy code is reliable if 

lim Pr[S„(Y") + S n ] = 0. 



(3) 



(4) 



A secrecy code is secret if the information leakage rate tends 
to zero as block length n — > oo, 



lim = 0. 



(5) 



Note that this leakage rate is not normalized by p. A nor- 
malized rate R is achievable if there exists a sequence of 



(2 npR , n) secrecy codes satisfying both Eqn. © and Eqn. ||5). 
The secrecy capacity C s is the supremum over all achievable 
rates. 

C. Outline 

To analyze the secrecy capacity C s , we first develop bounds 
as a function of the channel matrices Af, and A e . We then 
analyze these bounds for certain random matrices in the large 
system limit where nib/p — > pb and m e /p — >• p e as p — ► oo 
for fixed constants < p e < pb < 1/2. Lower bounds on the 
secrecy capacity, corresponding to Wyner's coding strategy for 
discrete memoryless channels are developed in Section III-AI 
Corresponding upper bounds are derived in Section III-B1 
Section lll-Cl provides an improved upper bound under a certain 
encoding constraint on Alice. Proofs are given in Section [ill] 

D. Notations 

For a matrix A £ W nxp and vector x £ {0, 1} P , we 
use A(x) to denote the matrix formed by concatenating the 
columns indexed by x, and we use A(i) to denote the ith 
column of A. Also, we use X% to denote the set of all 
binary vectors x 6 {0, 1} P with exactly k ones. We use 
H2{x) — — x\ogx — (1 — x) log(l — x) to denote binary the 
binary entropy function. We use log to denote the logarithm 
with base two and In to denote the logarithm with the natural 
base. 

II. Main Results 

Csiszar and Korner showed in [6| that the secrecy capacity 
of a discrete memoryless wiretap channel is given by 

C s = max [| /([/; Y) - \I{U; Z)] (6) 

where the auxiliary random variable U satisfies the Markov 
chain relationship: U — > X — > (Y, Z). It can be verified 
that this is also the secrecy capacity when the channels have 
discrete inputs and continuous outputs (see e.g. (5)). 

In some special cases, the secrecy capacity can be computed 
easily from (0. For example, if Ab and A e correspond to the 
first TOf, and m e rows of the pxp identity matrix respectively, 
then it is straightforward to show that 

C s = \ p p > . (7) 

I (J it m e > nib 

In this case, the secrecy capacity happens to be the difference 
of the individual channel capacities; thus as m e approaches rrib 
the secrecy capacity tends to zero. In the following sections 
we will develop bounds for a class of matrices inspired by 
compressed sensing. Interestingly, we will see that the secrecy 
behavior of these matrices differs greatly from the behavior 
shown in (0. 



A. Lower Bounds 

We say that a matrix A £ R mx P is fully linearly indepen- 
dent (FLI) if the span of each submatrix {^4(x) £ R mxm_1 : 
x £ X^ n _ l } defines a unique linear subspace of R m . Examples 
of FLI matrices include the first m rows of the pxp discrete 
cosine transform matrix or, with probability one, any matrix 
whose entries are drawn i.i.d. from a continuous distribution. 
A counter example is given by the first m rows of the pxp 
identity matrix. 

Our first result, which is proved in Section IIII-AI gives 
a general lower bound on the secrecy capacity for any FLI 
matrices. 

Theorem 1. Suppose that Ab and A e are fully linearly 
independent. If m e < nib, then the secrecy capacity is lower 
bounded by 

C^Ilog^-ilogdet^D 

+ E r^ l0Sdet (^^ e(x)Ae(x)T )- (8) 

The lower bound in Theorem [TJ is derived by evaluating the 
right hand side of (O when X is distributed uniformly over the 
set X^ lb _ 1 . We note that the condition m e < nib is necessary 
to obtain a nontrivial lower bound since the secrecy capacity 
may be equal to zero otherwise. 

Unfortunately, the bound in Theorem [TJ is difficult to com- 
pute if nib and p are large. One way to address this issue is to 
analyze the behavior for a random matrix (random matrices are 
denoted via boldface, uppercase letters). The following result 
is proved in Section IIII-BI 

Theorem 2. Suppose that Ab is fully linearly independent and 
A e is a random matrix whose elements are i.i.d. jV(0, 1). If 
m e < nib then the expectation of the secrecy capacity is lower 
bounded by 

EA^ s ]>|log( m f_J-^log(^) 

i=l 

where ip(x) = T' (x)/T(x) is Euler's digamma function. 

One benefit of Theorem [2] is that the bound is independent 
of the realization of the matrix A e and can be analyzed 
directly. An illustration of the bound is shown in Figure |2] 
as a function of m e /p for various values of p with nib/p held 
fixed. Remarkably, as p becomes large, the lower bound in 
Theorem [2] remains bounded away from zero for all values of 
m e strictly less than nib- This behavior is in stark contrast to 
the secrecy capacity shown in (01 ■ 

One shortcoming of Theorem [2] is that the bound holds 
only in expectation, and it is possible that it is violated for 
a constant fraction of matrices A e . The next result, which is 
proved in Section IIII-C1 shows that, in the asymptotic setting, 
the limit of the bound (O holds for almost every realization 
of A e . We use the notation {A& £ R m<p, xp} to denote a 
sequence of matrices indexed by the number of columns p. 
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Fig. 2. Illustration of the lower bound in Theorem[2]on the expected secrecy 
capacity Ea c [C s ] as a function of m e for various values of p when 
is fully linearly independent, A e is a random matrix whose elements are 
i.i.d. Af(0, 1), and m b /p = 0.2. 
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Theorem 3. Suppose that {A 
linearly independent matrices and {A, 
quence of random matrices whose elements are i.i.d. A/"(0, 1). 
Tjf me > ant/ mi /p — > Pb ant/ me/p Pe os 

p — >• oo where < p e < < 1/2, ?ne« //z<? asymptotic 
secrecy capacity is lower bounded by 



liminfa > £T 2 (p 6 ) - | 



-(P6-Pe)l0g(-^ 



(10) 



almost surely. 



Theorem [3] provides a concise characterization of the lower 
bound in the asymptotic setting. The bound is illustrated in 
Figure|2]in the case p = oo. Since the secrecy capacity can be 



equal to zero if m 



(p) 



Theorem [3] shows that there is a 



discontinuity in the asymptotic secrecy capacity as a function 
of p e . 

B. Upper Bounds via Channel Capacity 

This section considers the capacity of Bob's channel which 
is denoted Cb- We note that this capacity gives us an upper 
bound on the secrecy capacity. 

Upper bounding the capacity is more technically challeng- 
ing than lower bounding the secrecy capacity, since the optimal 
distribution on X may depend nontrivially on channel matrix 
Afj. The following result, which is proved in Section IIII-DI 
serves as a starting point. 

Theorem 4. If is fully linearly independent, then the 
channel capacity of Bob 's channel is upper bounded by 

C b < i max (log ( p .), max c(fc)) + ^fi£ (11) 



c(fc)= max if log (J-\\A b (iW) 



l<i<p 



max ilogdet(iA 6 (x)A fc (x) T ) 



(12) 



Although it is tempting to consider the expectation of (fTTT l 
with respect to a random matrix (as we did for Theorem 0, 
this is difficult since the maximization in ( fT2l occurs inside 
the expectation. 

Our next result, which is proved in Section IIII-EI leverages 
the strong concentration properties of the Gaussian distribution 
to characterize the asymptotic capacity for Gaussian matrices. 

Theorem 5. Suppose that {A^ G R m b P)xp } is a sequence 
of random matrices whose elements are i.i.d. Af(0, 1). If 
/p — > pb where < pb < 1/2, then the asymptotic 



» 



channel capacity of Bob's channel is given by 
lim C b = H 2 (p b ) 



(13) 



almost surely. 

Theorem [5] shows that the strategy used in our lower 
bounds, namely choosing X uniformly over j achieves 
the capacity of Bob's channel in the asymptotic setting. What 
is remarkable is that for this same input distribution, Eve learns 
very little about what is being sent, even if her channel matrix 
is equal to the first m e rows of Bob's channel matrix. 

C. Improved Upper Bound for a Restricted Setting 

We say that the distribution is symmetric if Pr[X = x] = 
Pr[X = x] for all x, x such that Yn=i x i = Y^!=i x i- The 
following result is proved in Section IIII-FI 

Theorem 6. Suppose that {A { b p] E R m i" )x P} and {A { e p) 6 
l£m<, p) xpj arg se q Uences of random matrices whose elements 
are i.i.d. 7V(0, 1). If m^ > mi and m£ /p — > p b , and 
m e p ^/p —> p e where < p e < p b < 1/2, and if Alice is 
restricted to use coding strategies that induce a symmetric 
distribution on X, then the asymptotic secrecy capacity is 
upper bounded by 



lim sup C s < H{X\WX + ^/p~b~/p~eV) 



(14) 



almost surely where X ~ Bernoulli(pb), W ~ A/"(0, 1) and 
V ~ A/"(0, 1) are independent random variables. 

The bound in Theorem |6] is strictly less than the channel 
capacity /^(Pfe) f° r a ll Pe > 0, and can be computed easily 
using numerical integration. We suspect that this result also 
holds without the symmetry restriction on X. 

D. Illustration of Bounds 

The bounds on the asymptotic secrecy capacity given in 
Theorems [3] are [6] and illustrated in Fig. [3] as a function of 
the size parameter p e of the eavesdropper channel. The bounds 
correspond to the setting where the elements of the matrices 
are i.i.d. Gaussian. Note that the lower bound on the secrecy 
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Fig. 3. Bounds on the asymptotic (normalized) secrecy capacity C s of the 
multiplicative Gaussian wiretap channel as a function of p e when pf, = 0.2 
and A;, and A e are random matrices whose elements are i.i.d. Af(0, 1). 
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Fig. 4. The (normalized) secrecy capacity C s of the multiplicative Gaussian 
wiretap channel as a function of p e when pj, = 0.2 and Af, and A e correspond 
to the first rows of the identity matrix. 



can be verified that both Z and Z|X have probability densities. 
(Note that the condition nib > m e is critical here, since Z does 
not have a density otherwise.) Thus we can write 



I(U; Z) = /(X; Z) = h(Z) - h(Z\X) 



(16) 



where h(-) denotes differential entropy (see e.g. Q). The 
entropy h(Z) can be upper bounded as 

h(Z) < max h{Z) 

Z : E[ZZ]=E[ZZ T ] 

< \ log ((2vre)™= det(E[ZZ T ])) (17) 
= |log((27re(^)r=det(A e ^)) (18) 

where ( fTTT i follows from the fact that the Gaussian distribution 
maximizes differential entropy and (fT8l follows from the fact 
that 



EfZZ^ 



(19) 



The conditional entropy ft,(Z|X) is given by 

ft(Z|X) = E [| log ((27re) m = det(A e (X)A e (X) T ))] (20) 

where we used the fact that, conditioned on any realization 
X = x, Z is a (non-degenerate) Gaussian random vector with 
covariance matrix A e (x.)A e (yL) T . Combining (Q3), (fT6i >. ( fT8l . 
and ( f20b with the expression of the secrecy capacity given in 
© completes the proof of Theorem Q] 

B. Proof of Theorem \2\ 

It is straightforward to show that A e is fully linearly 
independent with probability one. Since the secrecy rate is 
bounded, it thus follows from Theorem Q] and the linearity of 
expectation that 



capacity is nearly equal to that of the main channel for all 

Pe < Pb- 

For comparison, the secrecy capacity for the special case 
where At, and A e correspond to the first rows of the p x p 
identity matrix are shown in Fig. |4] In this case, the secrecy 
capacity is equal to the difference of the main channel capacity 
and eavesdropper capacity. 

III. Proofs 

A. Proof of Theorem Q] 

Let U = X where X is distributed uniformly over X^h-V 
Since A\, is fully linearly independent, the probability that Y 
is in the range space of Ai } (Sc) for any x e X^ _ x not equal 
to the true vector X is equal to zero. Thus, if(X|Y) = and 



7(l7;Y)=/(X;Y)=#(X) 



log 



p \ 



(15) 



Next, since A e is fully linearly independent and the number 
of nonzero values in X is strictly greater than the rank of A e , it 



EIC^MogL^J-^log^) 
-i-E[logdct(A e A^)] 



(- 7 i -^E[logdet(A e (x)A e (x) r )]. 



(21) 



Using well known properties of random Gaussian matrices 
(see e.g. El pp. 99-103]) shows that 



E[logdet(A e Aj)]=m e + loge^^(£=i±i) 

i=l 

E [ log dct (A e (x) A e (x) T )] = m e + log e J] V( 1 



' nib s 
2 / 



where the second equality holds for every x G X^ _ v 
Plugging these expressions into (|2~TT i completes the proof. 



C. Proof of Theorem \3\ 

Since A e is fully linearly independent with probability one, 
it is sufficient to consider the asymptotic behavior of the bound 
in Theorem [TJ If X is a random vector distributed uniformly 
over X% lb _ 1 then A e A^ and A e (X)A e (X) T are m e x m e 
Wishart matrices with p and mj — 1 degrees of freedom 
respectively. Using Lemma [6] in the Appendix gives 

lim ilogdet(iA e A^) = p(p e ) 

p— >OC ^ y 

lim ilogdet(-^ T A e (X)A^(X)) = Pbt i( Pe / Pb ) 
almost surely where ji(r) = (1 — r) In ( jhp) ~ rloge. Thus, 



lim 

p— >oo 



- E 



±)±logdet(A e A^) 



= lim 

p— >oo 



logdct(A e (x)A e (x) T ) 



Mogdet(A e A^) 



-±logdct(A e (X)A e (X) T ) 

= KPb) ~ PbP(pe/pb) 

= (l-p e )log( 7 ^-)-(p 6 



Pe) 



(22) 



(23) 



. l-p e ) ^-ru re/ - o \p b -p e 

almost surely where the substitution in (l22t is justified by 
the fact that the expectation of — log det(— A(X)A e (X) T ) 

TTtfy Tlfa 

with respect to both A and X is bounded uniformly for all p 
(see the proof of Theorem [2). 

Combining ( 123b with the well known fact that 



lim - log 



( P ) 

V m 6 — 1 / 



H 2 (p b 



(24) 



completes the proof of Theorem [3] 
D. Proof of Theorem [4] 

Let K = 53i=i -^t denote the number of ones in X. Then, 

7(X;Y)=J(X;Y|*r) + J(Y;X) (25) 

< I(X;Y\K) +logp (26) 

< max 7(X; YIA" = k) + \ogp (27) 

0<fc<p 

where (|25T > follows from the chain rule for mutual information, 
follows from the fact that I(Y; K) < H{k) < logp, and 
i follows from expanding the term I(X; Y\K). If we define 

c(k) = max J(X;Y) 

X6« 



then we have 



max/(X; Y) < max c(k) +logp. 

X 0<k<p 



(28) 



To complete the proof, we split the maximization over k into 
two cases. For < k < m b we use the simple bound 



max c(k) < max H (X) = log i 

0<fc<mi, XeA" fc : 0<fe<m 6 

For mb < k < p we use the following lemma. 



p ) 



Lemma 1. If mb < k < p, then c(fc) < c{k) where c{k) is 
given in (112t . 

Proof: Let X have any distribution on X% where m& < 
k < p. Since A e is fully linearly independent, both Y and 
Y|X have probability densities and we can write 



J(X;Y) = h(Y) - h{Y\X) 



(29) 



where h(-) denotes differential entropy (see e.g. Q). The 
entropy h(Y) can be upper bounded as 



h(Y) < max h(Y) 

Y : E[||Y|P]=E[||Y|| 2 ] 

= ^log(2^ e ^E[||Y|| 2 ]) 
< max 2* log (2ne£-JA b (i)\\ 



(30) 
(31) 



where (130t follows from the fact that an isotropic Gaussian 
vector maximizes differential entropy, and (|3~H follows from 
the fact that 

E[||Y|| 2 ]=E[E[||Y|| 2 |X]] 

< maxE[||Y|| 2 |X = x] 

X 

= maxtx(A b (x)A b (x) T ) 

X 

< max fcp b (*)|| 2 . 

1<1<P 

The conditional entropy h(Y\X.) is lower bounded by 

h(Y\X) = E [i log ((27re) m » det(A b (X)A(X) T ))] 

> min ilog((2 7 re) mi 'det(yl b (x)A 6 (x) T )) (32) 

where we used the fact that, conditioned on any realization 
X = x, Y is a (non-degenerate) Gaussian random vector with 
covariance matrix A b (K)A b (x.) T . Combining ( [29l , ( f3Tb and 
d32l i completes the proof of Theorem |4] ■ 

E. Proof of Theorem \5\ 

Since A e is fully linearly independent with probability one, 
it is sufficient to consider the asymptotic behavior of the bound 
in Theorem |4] The limit of the first term in the maximization 
is given by d24l) . To evaluate the second term, we use the 
following technical lemmas whose proofs are given in the 
Appendices [A] and iBl 



Lemma 2. 



limsup max — ||Af,(i)|| 2 < 1 

p^oo 1<«<P m b 



(33) 



almost surely. 

Lemma 3. If k > m b and k/p — > n where p b < k < 1, then 
liminf min i log det(±A h (x)A b (x) T ) > np(p b /n) (34) 

almost surely where p(r) = (1 — r) log (rzr) — rloge. 



The convergence show in Lemmas [2] and [3] leads immedi- 
ately to the following asymptotic upper bound on the term 
c(k) defined in ( IT2l >: 

limsup max §c(fc) 

p-foo m b <k<p p 

= ^Pfc log e 

almost surely. Since H 2 {pb) > ^P&loge for all p b G (0, 1/2), 
we conclude that the asymptotic capacity is upper bounded 
by H2(pb)- The achievable strategy outlined in the proof 
of Theorem Q] shows that i?2(p&) is also achievable which 
concludes the proof of Theorem [5] 

F. Proof of Theorem [6] 

Let K — Ysi=i Then, for any pair (U, X) such that 
17 -> X -> (Y, Z), we have 

I(U;Y) - I(U;Z) 

= /(X; Y) - /(X; Z) + J(X; Z|[7) - /(X; Y|[7) (35) 
<7(X;Y|7Q-7(X;Z|7Q 

+ /(X;Z|J7,ir)-7(X;Y|f7,K)+21ogp (36) 
< max A(Jfe) + 21ogp (37) 

0<k<p 

where 

A(fc) = 7(X; Y|7f = fc) - 7(X; Z|7sT = fc) 

+ 7(X;Z|f/,7C = fc) - 7(X; Y|£7, K = k). 

We now consider two cases. For the case < k < p, we 
use the upper bound 

AO) < 7(X; Y|7^ = k) + 7(X; Z\U, K = k) 
< 7(X; Y\K = k) + 7(X; Z\K = k) 

which follows from the non-negativity of mutual information 
and the data processing inequality. Following the steps out- 
lined in the proofs of Theorems |4] and [5] shows that 

limsup max ~ A(fc) < \{p h + p e ) log e < p b log e 

p— >oo XeA^J : mi,<k<p F 

(38) 

almost surely. (Note that this step does not require the sym- 
metry assumption.) 

Alternatively, for the case < k < mi,, we use the bound 

A(fc) < 7J(X|Z, K = k) + H(X\Y, K = k) 

which follows from the non-negativity of entropy and the 
fact that conditioning cannot increase entropy. Since is 
fully linearly independent almost surely, it follows from the 
proof of Theorem \T\ that 77 (X|Y, K = k) is equal to zero 
almost surely. To characterize the asymptotic behavior of the 
remaining term, 77(X|Z, K — k), we use the following lemma 
which is proved in Appendix ICl 



Lemma 4. Suppose that X is symmetric. If < k < mt, and 

k/p K where < K < pb, then 

limsupi77(X|Z,TT = fc) <g(n,p e ) (39) 

p— >oo 

almost surely where 

g(K, p e ) = H(X\WX + ^/p~ e V) (40) 

and X — Bernoulli(n), W ~ A/"(0, 1) and V — Af(0, 1) are 
independent random variables. 

Noting that g(K, p e ) is nondecreasing in k, we obtain the 
asymptotic upper bound 

limsupC s < max(g(pb,p e ),Pb)- (41) 

It can be verified numerically that this maximum occurs at 
g(pb 1 p e ) for all pb £ (0, 1/2) which completes the proof of 
Theorem [6] 

IV. Conclusion 

Bounds on the secrecy capacity C s for the multiplicative 
Gaussian wiretap channel were analyzed for a class of channel 
matrices inspired by compressed sensing. One natural exten- 
sion of the observations in this paper would be to consider the 
combined effects of additive noise in vector wiretap channels. 

Appendix 

A. Proof of Lemma [2] 

Note that the column magnitudes ||Af,(i)|| 2 , i = 1,2, • • • ,p 
are i.i.d. chi-square random variables with m& degrees of free- 
dom. Thus for any e £ (0, 1/2), the chi-square concentration 
inequality in Lemma [5] in Appendix D gives 

Pr[max ^||A 6 (i)|| 2 > 1 + e] < pexp(-^ \ PbP \ e 2 ) (42) 
which decays exponentially rapidly with p as p — > oo. 

B. Proof of Lemma \3\ 
For each x G X[ , let 

7V-(A 6 , x) = I log det (i A(x) A(x) T ) . (43) 

By the union bound, and the symmetry of A^ we have 

Pr [ min MA 6 ,x) < A < [ p \ p r [AT(A,x) < t], (44) 
L xe^ \kj 

for any arbitrary x G X?. Using the bound (?) < (pe/k) k 
and Lemma |6] in Appendix D, shows that for any e > 0, 

lim sup -ji- In Pr[7V( A, x) < p(p b / K ) - e] < -e 
which suffices to prove almost sure convergence. 



C. Proof of Lemma |4] Proof: We begin with a one-sided bound. For any r > 

Let X = -^-AjZ. Then, we have we have 

H(X\Z,K = k) <H{X\±,K = k) (45) Pr [n lndet (n W ) ^ *] 

p = Pr [In det (W) < nt + m log n] 

< ^ fl"(Xi|X, K = k) (46) = Pr 



i=l 



<^iJ(X l |X l ,K = fc) (47) 



dct(W)) > exp(— rnt + mlogn) 
< exp{rnt + rm log n) E ( det(W)) 



(55) 



where d55l > follows from Markov's inequality. If r < (n — 
where <g5} follows from the data processing inequality, (g5) ™) A then it can be shown (see e.g. pp. 99-103]) that 



follows from the chain rule and (|47| i follows from the fact that 
conditioning cannot increase entropy. 

Next, we observe that Xi can be written as 



E 

where 



det(W)) r = exp(-M(r)) 



1 m — 1 

= — E^W' ^0'))^ (48) M(r) = rmln(2) + y ri n r(^) - hr^ - 

l|A e « 



i=0 

W{Xi + <Ji(A e , X)V (49) if r i s an integer then we use the relation 



r 

where V ~ JV(0, 1) is independent of Wj.Xj and of (A e ,X) l n T(z) - lnTfz - r) = V ln(z - i) 

where ^— ' 



- l 2 (A e ,X) = -^^(A e (i) ) A e (i)) 2 X J -. (50) to 



obtain 



to. 



Using standard chi-square inequality, it is straightforward to M ( r ) ~~ rmln(2) + E ln ("2 ' ■?') 

show that 1=0 J=1 



m — 1 r 

i-2j ' 



lim max I |A ' (l)|i —11=0 (51) = rmlnn + V Vln (- 



i=0 j = l 



almost surely. With a bit more work, and the use of the fact Plugging this back into (El gives 
that, by the symmetry constraint, X is distributed uniformly 
over X? it can also be shown that 



m — 1 r 



1 ' lnPr[ilndet(iW)<t] < rnt- ^]n(2_±_?i 



lim max |af(A e ,X) --£-1=0 (52) i=o j=i 

p->oo l<i<p J Pe 

We now consider what happens as n — > 00. If r — Inn then 



almost surely. Thus, we conclude that the empirical distribu- . 

„ . / -rr vv \ 111 1 it is straightforward to show that 

tion of the pairs [Xi,Xi) converges weakly almost surely to 

the distribution on (X, WX + ^J n/ p e V), which concludes the 

proof. 



^ m — 1 r 

lim — \ > In 1 

77. — ^no ™ ' ' ' ' 



n-i-2j \ _ U (P) 

n->oo ni £ — ' L — ' " n log e ' 

i=0 j=l 6 



D. Technical Lemmas 



and thus 



Lemma 5 ( 11011 ). 7f X /s a chi-square random variable with 

n degrees of freedom then for all eg (0, 1 /2), lim sup In Pr [i In det (i W) < i] < t - 

n— yoo 

Pr[X > d(l + e)] < 

eX P \ 16 

rfe 2 . (53) 

To prove the other side of the bound, we use the same steps 
Lemma 6. Let W be an m x m Wishart random matrix with aS b e f° re to obtain 

n > m degrees of freedom. If m/n — > p G (0, 1] as n 00, 



then for any e>0, ' ' " ^ lnPr [\ Indet(IW) > f] < £ £ log (^) - rnt. 

limsup^— InPr riilogdet(iW) - > el < -t-^ T , , 

n-nxj nln " Ll " b v ™ ; j_ lo e e Letting r = In n leads to 



»=o j=i 



where \[ m sup i_ i n p r [I m det ( i W) > t] < ^ 

r nlnn Ln \n / — J — log e 

nging 
Lemma |6] 



f (1 - p) log (^-1 -ploge, if0<p<l 
u(p) = < ' . (54) Changing the base of the logarithms concludes the proof of 

l-loge, if P = 1 
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